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United  States  Senate 
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1 

1  - 1 

Dear  Mr.  Chairman: 


In  response  to  your  request,  we  are  reporting  to  you  on  the  effectiveness  of  the  Federal 
Aviation  Administration’s  management  of  computer  capacity  and  on  how  the  agency  can 
improve  its  practices  to  alleviate  problems  with  existing  systems.  Unless  you  publicly 
announce  the  contents  of  this  report  earlier,  we  plan  no  further  distribution  until  30  days 
from  the  date  of  this  letter.  At  that  time,  we  will  send  copies  to  the  Secretary  of 
Transportation;  the  Administrator,  Federal  Aviation  Administration;  the  Director,  Office  of 
Management  and  Budget;  interested  congressional  committees;  and  other  interested  parties. 
We  will  also  make  copies  available  to  others  upon  request. 


This  work  was  performed  under  the  direction  of  Jay  Etta  Z.  Hecker,  Director,  Resources, 
Community,  and  Economic  Development  Information  Systems.  Other  major  contributors  are 
listed  in  appendix  II. 


Sincerely  yours. 


Ralph  V.  Carlone 
Assistant  Comptroller  General 


I 


Eixecutive  Summary 


Purpose 


To  fulfill  its  key  missions  of  ensuring  safe  air  travel,  establishing  regula¬ 
tory  standards,  maintaining  security,  and  promoting  air  commerce,  the 
Federal  Aviation  Administration  (faa)  requires  a  vast  amount  of  infor¬ 
mation  technology  resources.  Computer  resources  are  especially  critical 
in  controlling  over  200,000  daily  flights  across  the  nation. 

Because  of  his  concern  about  the  potential  for  air  traffic  control  com¬ 
puter  systems  to  continue  to  experience  performance  shortfalls,  the 
Chairman,  Senate  Committee  on  Appropriations,  Subcommittee  on 
Transportation  and  Related  Agencies,  asked  gao  to  report  on  the  effec¬ 
tiveness  of  faa’s  agencywide  approach  to  computer  capacity  manage¬ 
ment  and  on  how  faa  can  improve  its  practices  to  alleviate  problems 
with  existing  systems. 


Background 


FAA  estimates  that  it  spends  about  $3  billion  annually  on  information 
technology  to  support  its  missions.  Much  of  this  is  expended  on 
numerous  computer  systems  to  promote  the  safe,  orderly,  and  expedi¬ 
tious  flow  of  civilian  and  military  aircraft.  Automated  systems  at  faa’s 
air  route  traffic  control  centers  and  Terminal  Radar  Approach  Control 
(tracon)  facilities  help  controllers  maintain  aircraft  separation.  Because 
these  systems  are  aging,  faa  plans  to  replace  them  with  the  Advanced 
Automation  System  during  the  latter  part  of  this  decade. 

FAA  also  has  12  “Common  System”  computer  facilities  that  provide  gen¬ 
eral-purpose  data  processing  for  mission  and  administrative  areas,  such 
as  aviation  safety,  and  financial,  material,  and  human  resources,  faa 
plans  to  phase  out  these  computer  facilities  in  1995,  relying  instead  on 
computer  resources  owned  and  operated  by  a  contractor,  faa’s  research, 
engineering,  and  development  program,  an  important  element  in 
improving  air  traffic  control  operations  and  aviation  safety  and 
security,  is  also  supported  by  computer  systems. 

faa’s  tremendous  investment  in  information  technology  resources 
requires  effective  management  of  computer  capacity.  Capacity  manage¬ 
ment  is  the  process  by  which  the  components  of  a  computerized  system 
are  configured,  used,  and  maintained  to  effectively  and  efficiently  pro¬ 
cess  work  loads;  it  provides  the  basis  for  predicting  when  hardware  and 
software  need  to  be  upgraded  to  meet  projected  growth. 
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Over  the  past  2  years,  gao  has  issued  several  reports  on  faa’s  computer 
capacity  management  practices.*  For  instance,  gao  found  that  the  lack  of 
a  computer  capacity  management  program  for  air  traffic  control  auto¬ 
mation  systems  resulted  in  computer  capacity  shortfalls  at  several  facil¬ 
ities,  as  well  as  the  inability  to  accurately  determine  future  computer 
capacity  requirements.  Further,  without  a  capacity  management  pro¬ 
gram,  FAA  could  not  determine  the  cause  of  performance  problems  for 
the  Common  System  computers  and  was  unable  to  accurately  project 
future  capacity  requirements. 

Results  in  Brief 

I 

Management  of  computer  capacity  is  critical  to  faa’s  meeting  its  mis¬ 
sions,  such  as  ensuring  safe  air  travel.  Although  faa  has  recently  made 
some  limited  improvements,  it  has  not  implemented  a  comprehensive 
capacity  management  program  for  its  major  automated  systems  because 
such  a  program  is  not  a  priority.  As  a  result,  faa  lacks  adequate  com¬ 
puter  capacity  management  policies,  procedures,  expertise,  and  tools. 
Without  a  comprehensive  program,  faa  does  not  know  how  long  current 
systems,  such  as  those  used  to  assist  controllers  in  separating  aircraft, 
will  continue  to  meet  capacity  requirements,  nor  does  it  know  its  future 
capacity  needs. 

Principal  Findings 

Capacity  Management  Is 
Critical  to  Achieving 
Mission  Success 

Capacity  management  is  critical  to  faa’s  mission  of  providing  essential 
safety  and  aviation  services.  For  example,  capacity  management  is  nec¬ 
essary  to  correct  or  prevent  processing  shortfalls  that  could  hinder  con¬ 
trollers’  ability  to  maintain  safe  separation  of  aircraft.  Capacity 
management  has  two  key  components:  (1)  performance  management, 
which  measures  and  evaluates  system  performance  to  prevent  or  cor¬ 
rect  problems;  and  (2)  capacity  planning,  in  which  required  resources 
are  projected  based  on  estimated  work  loads  and  reserve  capacity.  Fed¬ 
eral  regulations  and  related  guidance  require  agencies  to  conduct 
capacity  management  activities.  Further,  private  sector  organizations 

'Air  Traffic  Control:  Computer  Capacity  Shortfalls  May  Impair  Flight  Safety  (GAO/IMTEC-89-63, 
July  6,  rnSQ!;  FAA  Ph-ocurement:  Major  Data-Processing  Contract  Should  Not  Be  Awarded  (GAO/ 
IMTEC-90-38,  May  25,  1990);  Air  Traffic  Control:  Smaller  Terminal  Systems’  Capacity  Requirements 
Need  to  be  Defined  (GAO/IMTEC-90-50,  June  25,  1990);  Air  Traffic  Control:  Inadequate  Planning 
Increases  Risk  of  Computer  Failures  in  Los  Angeles  (GAO/IMTEC-90-49,  July  16,  1990). 
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believe  capacity  management  is  an  important  component  of  their  infor¬ 
mation  resource  management  activities  because  it  allows  them  to  suc¬ 
cessfully  meet  current  and  future  capacity  requirements. 


FAA  Does  Not  Perform 
Key  Capacity  Management 
Activities 


FAA  does  not  have  complete  capacity  management  data  to  support  its 
decisions  on  operating,  upgrading,  or  replacing  systems,  resulting  in 
decisions  that  are  sometimes  incorrect  or  untimely.  For  example,  faa  did 
not  take  sufficient  steps  to  upgrade  terminal  air  traffic  control  automa¬ 
tion  systems  until  capacity  shortfalls  occurred,  which  impaired  control¬ 
lers’  ability  to  maintain  safe  separation  of  aircraft,  faa  monitors  the 
performance  and  utilization  of  some  of  its  systems  only  periodically  or 
in  response  to  capacity-related  problems.  Other  faa  systems  are  moni¬ 
tored  more  consistently,  but  not  in  sufficient  detail  to  allow  effective 
problem  identification  and  correction. 


Given  the  previous  capacity  shortfalls  at  several  air  traffic  control  facil¬ 
ities  and  continued  delays  in  developing  and  deploying  the  Advanced 
Automation  System,  accurately  predicting  future  work  loads  for  air 
traffic  control  systems  is  essential.  However,  except  for  newly  estab¬ 
lished  capacity  planning  activities  concerning  the  Common  System, 
capacity  planning  has  been  almost  non-existent  at  faa.  Although  several 
one-time  capacity  planning  studies  were  conducted  for  systems  that 
needed  to  be  upgraded  or  replaced,  planning  activities  are  not  per¬ 
formed  regularly. 


Lack  of  Policies, 
Procedures,  Expertise,  and 
Tools  Hinder  Effective 
Capacity  Management 


faa’s  inconsistent  performance  management  and  inadequate  capacity 
planning  are  due  to  faa’s  failure  to  make  capacity  management  a  pri¬ 
ority.  As  a  result,  faa  lacks: 

agencywide  guidance  on  computer  capacity  management  for  its  major 
automated  systems; 

system-specific  capacity  management  procedures  for  its  various  air 
traffic  control  and  administrative  systems; 

capacity  management  positions  and  expertise  to  ensure  that  an  effective 
capacity  management  program  is  in  place;  and 

consistent  implementation  of  automated  performance  management  and 
capacity  planning  tools  to  assist  in  the  collection,  analysis,  prediction, 
and  modelling  of  current  or  future  capacity  management  data. 
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Recommendations 


Agency  Comments 


GAO  recommends  that  the  Administrator,  faa: 

•  Develop  computer  capacity  management  policies  and  procedures  for  the 
air  traffic  control  systems  and  Common  System.  These  policies  and  pro¬ 
cedures  should  require  the  implementation  of  capacity  management  pro¬ 
grams  for  each  type  of  system  and  include  accepted  performance 
management  and  capacity  planning  activities. 

•  Obtain  or  develop  the  additional  capacity  management  expertise  neces¬ 
sary  to  develop  policies  and  procedures  and  administer  capacity  man¬ 
agement  programs. 

•  Acquire  or  develop  automated  performance  management  and  capacity 
planning  tools  for  each  system  and  consistently  implement  them  to 
assist  in  the  collection,  analysis,  and  reporting  of  performance,  utiliza¬ 
tion,  and  capacity  data. 


Department  of  Transportation  and  faa  officials  acknowledged  that 
capacity  management  problems  exist,  but  stated  that  faa  is  taking 
actions  to  correct  them.  Officials  added  that  the  report  needs  to  more 
clearly  identify  differences  between  real-time  air  traffic  control  and  gen¬ 
eral-purpose  systems  because  capacity  management  activities  for  these 
two  types  of  systems  vary  significantly,  faa  officials  also  stated  that 
capacity  shortfalls  in  air  traffic  control  computers  have  not  caused 
unsafe  separation  of  aircraft  because  controllers  have  other  means,  such 
as  primary  radar  information,  to  ensure  that  safety  is  not  compromised. 

We  recognize  in  the  report  faa’s  actions  to  improve  capacity  manage¬ 
ment.  However,  because  capacity  management  is  a  crucial  function  in 
helping  meet  the  agency’s  missions,  faa  needs  to  implement  a  compre¬ 
hensive  computer  capacity  management  program.  Regarding  the  differ¬ 
ences  between  real-time  and  general-purpose  systems,  we  acknowledge 
in  our  recommendations  that  specific  programs  need  to  be  tailored  to 
each  type  of  system.  However,  the  capacity  management  framework 
presented  in  the  report  is  applicable  to  both  types  of  systems.  Finally, 
while  we  do  not  have  evidence  that  capacity  shortfalls  have  actually 
compromised  flight  safety,  we  have  previously  reported  that  such 
shortfalls  do  impair  controllers’  ability  to  maintain  separation  of  air¬ 
craft  and  therefore  may  impair  flight  safety. 
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Introduction 


The  Federal  Aviation  Administration  (faa)  is  responsible  for  ensuring 
safe  air  travel,  establishing  regulatory  standards,  maintaining  security, 
and  promoting  air  commerce.  To  fulfill  these  key  missions,  extensive  use 
of  information  processing  and  communications  is  necessary.  Computer 
and  communications  resources  are  especially  critical  to  faa  in  control¬ 
ling  over  200,000  daily  flights  throughout  the  nation. 


Air  Traffic  Control 
Automation  Systems 


faa’s  air  traffic  control  mission  is  to  promote  the  safe,  orderly,  and 
expeditious  flow  of  civilian  and  military  aircraft.  To  accomplish  this 
mission,  air  traffic  controllers  communicate  weather  information, 
instructions,  and  clearances  to  pilots  and  other  personnel;  maintain  safe 
distances  between  aircraft;  and  guide  aircraft  departures  and 
approaches.  Several  computer  and  communication  systems  assist  air 
traffic  controllers  in  performing  these  functions. 


To  control  air  traffic  that  is  en  route  between  airports,  faa  maintains  20 
air  route  traffic  control  centers  in  the  continental  United  States.  Each  of 
these  centers  is  supported  by  an  automated  system,  known  as  the  Host 
computer  system,  to  help  air  traffic  controllers  maintain  aircraft 
separation. 


FAA  also  maintains  182  Terminal  Radar  Approach  Control  (tracon)  facil¬ 
ities  to  sequence  and  separate  aircraft  arriving  at  or  departing  from  air¬ 
ports  under  its  control.  The  New  York  facility,  which  provides  air  traffic 
control  services  for  the  New  York  metropolitan  airspace,  is  the  busiest 
TRACON  in  the  country  and  is  supported  by  a  unique  system  known  as  the 
Automated  Radar  Terminal  System  HIE.  faa’s  62  other  large  tracons  are 
supported  by  Automated  Radar  Terminal  System  IIIAs,  while  the 
119  smaller  tracons  are  supported  by  Automated  Radar  Terminal 
System  IIAs. 

The  Advanced  Automation  System,  the  centerpiece  of  faa’s  National 
Airspace  System  Modernization  plan,  is  scheduled  to  replace  air  route 
traffic  control  center  and  tracon  systems  with  new  hardware,  software, 
and  controller  workstations,  faa  believes  that  the  advanced  system, 
scheduled  for  initial  implementation  in  the  mid  to  late  1990s,  will 
increase  controller  productivity,  reduce  operating  costs,  save  fuel  and 
passenger  time,  and  allow  controllers  to  handle  anticipated  traffic 
increases  more  safely  and  efficiently.  However,  as  we  have  previously 
reported,  recent  delays  in  the  Advanced  System  will  force  faa  to  operate 
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aging  computer  systems  at  its  air  traffic  control  facilities  for  several 
more  years.' 


General-Purpose  Data- 
Processing  Systems 


faa’s  general-purpose  data-processing  systems,  known  as  the  Common 
System,  consist  of  23  computers  at  12  agency  facilities.  This  system  pro¬ 
vides  support  for  mission  and  administrative  areas,  such  as  airport  and 
aviation  activity  (excluding  real-time  air  traffic  control);  aviation 
safety;  national  airspace  facilities;  and  financial,  material,  and  human 
resources. 


FAA  maintains  that  it  is  no  longer  possible  or  desirable  to  upgrade  the 
Common  System  to  meet  the  agency’s  rapidly  growing  data-processing 
needs.  Therefore,  it  developed  the  Computer  Resources  Nucleus  project 
to  replace  the  Common  System,  faa  plans  to  procure  data-processing 
services  from  a  contractor  who  will  be  responsible  for  providing,  main¬ 
taining,  and  operating  the  computer  facilities,  equipment,  system 
software,  and  technical  support  services  to  meet  the  agency’s  general- 
purpose  data-processing  needs  over  the  next  10  years,  faa  plans  to 
award  this  contract  in  early  1992  and  anticipates  turning  the  entire 
Common  System  over  to  the  contractor  by  1995. 


Scientific  and 
Engineering  Computer 
Systems 


In  addition  to  air  traffic  control  and  general-purpose  computer  systems, 
FAA  supports  systems  for  research,  engineering,  and  technical  develop¬ 
ment  programs.  Scientific  and  engineering  computers  are  used  by  the 
FAA  Technical  Center,  located  at  the  Atlantic  City,  New  Jersey  Interna¬ 
tional  Airport,  to  assist  in  research  projects  covering  topics  such  as  air¬ 
craft  separation  standards,  transport  crashworthiness,  and  aircraft 
icing.  FAA  recently  decided  to  phase  out  these  computers  and  use  other 
available  systems.  One  computer  was  decommissioned  in  July  1991, 
while  the  other  will  continue  to  operate  until  remaining  programs  can  be 
transported  to  either  the  Computer  Resources  Nucleus  project  or  other 
agency  computer  systems. 


1  Air  Traffic  Control:  Continuing  Delays  Anticipated  for  the  Advanced  Automation  System  (GAO/ 
IMTEC-90-63,  July  18.  1990);  Delays  in  Critical  Air  Traffic  Control  Modernization  Projects  Require 
Increased  FAA  Attention  to  Existing  Systems  (GAO/T-IMTEC-91-14,  June  14,  1991). 


Page  9 


GAO/IMTEC-92-2  FAA’s  Capacity  Management  Program 


Chapter  1 
Introduction 


Prior  Reports 
Highlight  FAA’s 
Computer  Capacity 
Management  Problems 


Improvements  in 
FAA’s  Capacity 
Management 


Over  the  past  2  years,  we  have  issued  several  reports  on  faa’s  ineffec¬ 
tive  computer  capacity  management  practices.  In  1989,  we  reported  that 
the  lack  of  a  computer  capacity  management  program  resulted  in  com¬ 
puter  capacity  shortfalls  at  large,  busy  tracon  facilities.^  These 
shortfalls  impaired  controllers’  ability  to  maintain  safe  separation  of 
aircraft.  Many  facilities  reported  that  they  experienced  instances  of  air¬ 
craft  position  and  identification  information  disappearing  from  control¬ 
lers’  displays,  data  flickering  on  displays,  and  delayed  computer 
responses  to  controllers’  attempts  to  update  or  request  data.  Last  year 
we  reported  that  faa  did  not  know  if  its  current  automation  plan  for  the 
consolidated  Los  Angeles  basin  tracon  facility  would  meet  future  needs 
because  the  agency  lacked  a  computer  capacity  management  program.^ 
We  also  reported  that,  for  its  smaller  tracon  facilities,  faa  (1)  had  no 
assurance  that  an  ongoing  system  upgrade  would  meet  capacity  needs, 
and  (2)  could  not  adequately  determine  future  capacity  requirements.^ 

Regarding  the  Common  System,  we  previously  reported  that  the  lack  of 
a  capacity  management  program  resulted  in  insufficient  information  to 
determine  if  response  time  problems  were  caused  by  inadequate 
capacity .5  Further,  faa  was  unable  to  adequately  project  future  capacity 
requirements  and  substantiate  its  justification  for  the  Computer 
Resources  Nucleus  project. 


In  response  to  our  previous  recommendations,  faa  has  taken  steps  to 
prevent  capacity  shortfalls  from  occurring.  For  example,  at  large 
TRACONS,  FAA  is  replacing  existing  computer  memories,  which  employ 
outdated  technology,  with  high-speed,  solid-state  memories  and  opti¬ 
mizing  operational  software  to  improve  execution  time.  The  agency  also 
plans  to  further  increase  computer  capacity  by  procuring  additional 
processors  to  expand  large  tracon  system  configurations  to  their  max¬ 
imum  design  limits. 


^Air  Traffic  Control:  Computer  Capacity  Shortfalls  May  Impair  Flight  Safety  fGAO/IMTEC-89-6,S 
July  6,  1989).  “ - ^ 

®Air  Traffic  Control:  Inadequate  Planning  Increases  Risk  of  Computer  Failures  in  Los  Angeles  fGAO/ 
IMTEC-90-49,  July  16,  1990), - ^ - 

'‘Air  Traffic  Control:  Smaller  Terminal  Systems’  Capacity  Requirements  Need  to  be  Defined  (GAO/ 
IMTEC-90-.50,  June  25,  1990).  - - 

®FAA  Procurement:  Major  Data-Processing  Contract  Should  Not  Be  Awarded  fGAO/IMTEC-90-,S8 
May  25, 1990).  - 
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To  better  manage  computer  capacity,  faa  has  initiated  some  limited 
activities,  such  as  routinely  capturing  data  on  system  work  load  (e.g., 
number  of  tracks,  number  of  targets),  processor  utilization,  and  memory 
utilization  of  its  large  tracon  systems.  Additionally,  faa  plans  to  award 
a  contract  later  this  year  for  the  development  of  an  integrated  perform¬ 
ance  monitoring  analysis  system  designed  to  improve  the  collection, 
analysis,  and  reporting  of  air  traffic  control  performance  and  capacity 
data.  FAA  officials  added  that  this  contract  and  related  activities  will 
represent  the  foundation  of  its  future  capacity  planning  activities  for  air 
traffic  control  systems. 


Objective,  Scope,  and 
Methodology 


Our  objective  was  to  determine  the  effectiveness  of  the  agency’s 
approach  to  computer  capacity  management  and  identify  how  faa  can 
improve  its  practices  to  alleviate  problems  with  existing  systems. 

To  accomplish  our  objective,  we  reviewed  federal  regulations  and  identi¬ 
fied  criteria  for  computer  capacity  management  programs.  Because 
these  regulations  did  not  provide  detailed  capacity  management  criteria, 
we  used  additional  federal  guidance  to  augment  the  regulations.  To  fur¬ 
ther  expand  upon  this  criteria  and  obtain  a  broader  perspective  on 
capacity  management  principles  and  practices,  we  reviewed  industry 
publications,  reports,  and  conference  proceedings.  We  also  interviewed 
officials,  analyzed  capacity  management  reports  and  studies,  and  identi¬ 
fied  effective  capacity  management  practices  from  several  private 
sector  organizations  recognized  for  implementing  effective  capacity 
management  programs.  We  selected  these  private  concerns  based  on  our 
review  of  industry  publications  and  on  reconunendations  from  industry 
experts. 

On  the  basis  of  the  criteria  we  identified  and  synthesized,  we  assessed 
faa’s  capacity  management  practices.  We  selected  seven  systems  for 
review,  covering  both  air  traffic  control  and  general-purpose  systems, 
on  the  basis  of  previous  reports  citing  computer  capacity  management 
problems.  In  reviewing  these  systems,  we  analyzed  agency  and  con¬ 
tractor  capacity  plans,  studies,  reports,  and  written  guidance  describing 
existing  and  planned  capacity  management  practices.  We  interviewed 
FAA  officials  to  determine  existing  capacity  management  practices  and 
compliance  with  federal  guidance  and  accepted  computer  capacity  man¬ 
agement  practices. 

In  addition,  we  evaluated  agency  wide  capacity  management  policy,  and 
discussed  performance  management  and  capacity  planning  activities 
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with  FAA  officials  from  headquarters,  regional  offices  and  centers,  and 
air  traffic  control  facilities.  We  selected  these  locations  because  they 
either  (1)  maintained  a  large  amount  of  the  agency’s  computer 
resources,  (2)  had  computer  systems  representative  of  other  comparable 
facilities,  or  (3)  contained  agency-unique  computer  processing  charac¬ 
teristics.  We  also  updated  our  prior  work  on  faa  capacity  management 
and  identified  the  agency’s  actions  in  response  to  our  recommendations. 
In  addition,  we  identified  reasons  for  discrepancies  between  faa’s 
capacity  management  activities  and  our  criteria. 

We  performed  our  work  at  faa  headquarters  in  Washington,  D.C.;  the 
FAA  Technical  Center  at  the  Atlantic  City,  New  Jersey  International  Air¬ 
port;  the  Mike  Monroney  Aeronautical  Center  in  Oklahoma  City, 
Oklahoma;  faa  regional  offices  in  Jamaica,  New  York;  Kansas  City,  Mis¬ 
souri;  and  Atlanta,  Georgia;  tracon  facilities  in  Pensacola,  Florida; 
Tampa,  Florida;  Atlanta,  Georgia;  Philadelphia,  Pennsylvania;  and 
Westbury,  New  York;  and  air  route  traffic  control  centers  in  Miami, 
Florida;  Hampton,  Georgia;  Nashua,  New  Hampshire;  and  Ronkonkoma, 
New  York.  We  also  conducted  work  at  Farmland  Industries,  Inc.  in 
Kansas  City,  Missouri;  MCI  Corporation  in  Arlington,  Virginia;  and 
Yellow  Freight  Systems  in  Overland  Park,  Kansas. 

Our  review  was  performed  between  August  1990  and  October  1991,  in 
accordance  with  generally  accepted  government  auditing  standards.  The 
views  of  faa  officials  were  obtained  on  the  key  facts,  conclusions,  and 
recommendations  contained  in  this  report.  Their  views  are  incorporated 
throughout  the  report  as  appropriate.  In  addition,  we  obtained  oral  com¬ 
ments  from  Department  of  Transportation  and  faa  officials  on  a  draft  of 
this  report.  These  comments  and  our  analysis  are  also  included  in  the 
report. 
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Computer  capacity  management  at  faa  is  critical  to  the  safe  and  effi¬ 
cient  use  of  the  nation’s  airspace.  Capacity  management  is  the  process 
by  which  the  components  of  a  computerized  system  are  configured,  uti¬ 
lized,  and  maintained  to  effectively  and  efficiently  process  work  loads. 
Additionally,  it  provides  the  analytical  basis  for  predicting  when  hard¬ 
ware  and  software  configurations  will  need  to  be  upgraded  to  meet  pro¬ 
jected  work-load  growth.  The  Federal  Information  Resources 
Management  Regulation  and  related  guidance  require  government  agen¬ 
cies  to  conduct  capacity  management  activities  in  planning,  acquiring, 
and  using  computer  resources.  Capacity  management  has  two  key  com¬ 
ponents:  (1)  performance  management,  dedicated  to  measuring  and 
evaluating  current  utilization  of  computer  resources;  and  (2)  capacity 
planning,  dedicated  to  determining  what  additional  capacity  is  needed  to 
support  future  missions  and  ensuring  that  it  is  available  when  needed. 
Private  sector  organizations  consider  capacity  management  activities  to 
be  high-priority  company  programs  and  stress  their  criticality  to  busi¬ 
ness  success. 


Performance 
Management 
Necessary  to  Meet 
Current  Needs 


Performance  management  enables  an  agency  to  provide  the  best  pos¬ 
sible  service  to  users  on  the  existing  system.  Federal  guidance  empha¬ 
sizes  the  need  for  performance  management  to  ensure  that  services  are 
provided  efficiently  and  effectively.  Key  performance  management 
activities  include:  (1)  establishing  performance  objectives,  (2)  measuring 
overall  system  performance  and  individual  component  utilization,  and 
(3)  analyzing  and  reporting  performance  data. 

Establishing  performance  objectives  involves  defining  acceptable  levels 
of  service  for  system  users.  Levels  of  service  are  expressed  in  terms 
such  as  response  time,  job  completion  or  turnaround  time,  availability 
and  reliability  of  computer  equipment  and  services,  cost  of  operation, 
and  accuracy  and  quality  of  output.  Organizations  should  formalize 
their  performance  objectives  by  establishing  service-level  agreements 
between  system  users  and  system  providers.  Through  service-level 
agreements,  users  precisely  state  their  system  performance  expecta¬ 
tions,  system  providers  formally  commit  to  satisfy  these  expectations, 
and  both  parties  agree  on  the  approach  to  measure,  report,  and  evaluate 
performance.  These  agreements  should  be  updated  periodically  to 
ensure  current  objectives  are  being  met. 

Once  the  performance  objectives  have  been  defined,  performance  and 
utilization  data  should  be  collected.  Such  performance  measurement 
should  focus  on  both  the  system  as  a  whole  as  well  as  the  individual 
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system  components.  The  components  of  a  system  include  the  central 
processor,  memory,  input/output  channels,  peripheral  devices,  commu¬ 
nications  processors,  and  the  associated  software  and  data  files.  Because 
system  components  have  different  operating  characteristics  (e.g.,  size 
and  speed),  and  because  the  demand  upon  these  components  varies  with 
the  work  load,  poor  performance  management  can  result  in  bottlenecks 
that  degrade  overall  system  performance.  At  a  minimum,  performance 
and  utilization  data  should  be  collected  for  the  central  processor, 
memory,  and  channels.  This  data  can  be  gathered  by  system  facilities, 
hardware  monitors,  and  software  monitors. 

After  performance  and  utilization  data  are  gathered,  they  should  be 
analyzed  to  determine  whether  desired  performance  objectives  are  being 
achieved,  to  isolate  the  causes  of  problems,  and  then  recommend  adjust¬ 
ments.  These  adjustments  may  include  changing  operational  procedures 
(e.g.,  processing  certain  work  loads  at  different  times  of  the  day),  aug¬ 
menting  hardware,  and  making  changes  to  operating  system  parameters. 
Further,  performance  and  utilization  data  should  be  retained  for  later 
analysis  to  ascertain  trends  or  determine  the  impact  of  significant 
events,  such  as  the  introduction  of  new  applications. 


Performance  management  data  provides  a  baseline  to  predict  future 
resource  needs  through  capacity  planning.  Capacity  planning  is  the  pro¬ 
cess  of  predicting  future  work  loads  and  determining  what  system 
resources  will  be  needed  to  process  them  effectively.  Capacity  planning 
also  provides  analytical  data  to  support  the  procurement  of  system 
additions  and  enhancements.  The  capacity  planning  process  includes  (1) 
projecting  future  work  loads  and  required  user  service  levels,  (2)  pro¬ 
posing  resources  to  meet  these  demands,  and  (3)  planning  to  obtain  the 
required  resources. 

Projecting  future  work  loads  and  service  levels  requires  understanding 
the  current  work  load  and  determining  what  changes  are  likely  to  occur. 
For  example,  the  use  of  certain  applications  may  increase  at  some  pre¬ 
dicted  rate,  while  use  of  other  applications  may  decrease  or  disappear. 
Further,  new  applications  may  be  implemented  on  a  given  schedule  and 
user  service-level  expectations  may  change  as  technology  changes.  It  is 
generally  more  difficult  to  accurately  predict  long-term  work-load 
changes  (6  to  10  years)  than  short-term  work-load  changes  (less  than  1 
year). 


Capacity  Planning 
Essential  for  Meeting 
Future  Processing 
Requirements 
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Once  future  work  loads  and  user  service-level  projections  are  deter¬ 
mined,  systems  that  will  satisfy  these  requirements  can  be  proposed. 
Several  methods,  including  modelling  and  pilot  testing,  can  be  used  to 
define  and  evaluate  alternative  system  configurations  that  will  process 
the  future  work  load  effectively.  Once  the  best  alternative  is  identified, 
plans  for  obtaining  the  required  system  resources  on  a  specified 
schedule  are  needed.  Capacity  plans  help  agencies  ensure  that  equip¬ 
ment  is  available  to  meet  actual  need,  thereby  preventing  processing  and 
capacity  shortfalls.  Capacity  plans  should  be  reviewed  and  updated  at 
least  annually. 


Private  Sector 
Organizations  Cite 
Necessity  of  Computer 
Capacity  Management 
Programs 


Private  sector  organizations  operate  in  an  intensely  competitive  business 
environment  where  the  efficient  and  effective  use  of  information 
resources  is  essential  to  success.  Private  organizations  we  surveyed 
agree  that  capacity  management  is  a  critical  component  of  their  infor¬ 
mation  resource  management  activities.  Such  organizations  have  devel¬ 
oped  and  implemented  comprehensive  capacity  management  programs 
that  allow  them  to  successfully  meet  current  and  future  capacity 
requirements. 


At  Farmland  Industries,  an  agricultural  cooperative,  capacity  manage¬ 
ment  practices  include  establishing  performance  objectives  between 
system  users  and  data  processing  personnel.  Farmland’s  capacity  man¬ 
agement  team  regularly  monitors  system  performance  and  produces 
weekly  and  monthly  performance  reports  that  include  central  processor 
utilization,  disk  capacity,  and  response  time.  These  performance  reports 
are  reviewed  by  system  managers  to  assist  in  their  system  capacity 
assessments.  In  addition,  system  users  are  interviewed  periodically  to 
update  work-load  projections. 

Covia  Corporation,  an  airline  reservation  company,  also  employs  a 
formal  approach  for  managing  its  computer  capacity.  An  important  ele¬ 
ment  of  Covia’s  program  is  the  construction  of  capacity  planning  reports 
that  support  its  overall  strategic  planning  process.  According  to  a  Covia 
capacity  planner,  the  firm’s  rapid  growth  in  the  industry  during  the  past 
2  decades  was  made  possible  by  the  expansion  of  its  data  processing 
capabilities.  He  stated  that  Covia’s  computer  capacity  management  pro¬ 
gram  directly  contributed  to  its  business  success. 


In  a  similar  fashion,  MCI  Corporation,  a  telecommunications  organiza¬ 
tion,  has  successfully  employed  capacity  management  programs  for 
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both  its  administrative  data  processing  and  telephone  network  opera¬ 
tions.  In  particular,  MCI  uses  numerous  software  tools  to  help  collect 
system  performance  and  capacity  data.  Company  managers  indicated 
that  when  conducting  performance  measurement  and  data  analysis 
activities,  approximately  10  percent  of  the  time  is  spent  collecting  data, 
and  up  to  90  percent  of  time  is  devoted  to  data  analysis.  This  has 
improved  MCI’s  ability  to  correct  capacity  problems  and  avoid  predicted 
problems,  according  to  an  MCI  capacity  specialist. 

Overall,  several  private  sector  firms  have  found  that  their  investment  in 
capacity  management  has  allowed  them  to  compete  successfully  today 
while  planning  better  for  the  future.  They  recognize  that  capacity  man¬ 
agement  is  pivotal  to  meeting  their  business  goals.  Appendix  I  provides 
additional  details  on  several  private  firms  and  their  capacity  manage¬ 
ment  programs. 
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Performance 
Management  Efforts 
Are  Insufficient 


Although  FAA  has  recently  improved  its  capacity  management  practices 
in  some  areas,  it  has  not  implemented  effective  computer  capacity  man¬ 
agement  practices  for  most  of  its  automated  systems.  Specifically,  per¬ 
formance  objectives  for  many  systems  are  not  current,  measurement  of 
system  performance  is  inconsistent,  and  the  analysis  and  reporting  of 
this  data  are  limited.  In  addition,  future  work  loads  are  not  projected, 
additional  resources  are  not  proposed,  and  capacity  plans  are  not  pre¬ 
pared  for  many  systems.  As  a  result,  faa  does  not  know  how  well  its 
systems  are  performing,  when  it  can  anticipate  capacity  and  perform¬ 
ance-related  problems,  or  how  it  will  address  these  problems. 


faa’s  performance  management  activities  for  each  of  its  automated  sys¬ 
tems  need  improvement.  Performance  objectives  for  several  automated 
systems  have  been  established,  but  some  have  not  been  kept  up  to  date. 
Further,  some  faa  systems  are  monitored  only  periodically  or  after  a 
capacity-related  problem  occurs,  and  the  type  and  amount  of  perform¬ 
ance  and  utilization  data  collected  and  reported  varies.  The  following 
table  summarizes  faa’s  compliance  with  the  performance  management 
criteria  presented  in  chapter  2  for  each  of  the  agency’s  major  systems. 


Performance  Management  Actlvltes 

Establishing 

Performance 

Objectives 

Measuring  Performance 
and  Resource 

Utilization 

Analyzing  and 

Reporting  Data 

Air  Traffic  Control  Systems 

Air  Route  Traffic  Control  Center 

0 

0 

New  York  TRACON 

0 

0 

Large  TRACONs 

d 

0 

Small  TRACONs 

0 

O 

Common  System 

Aeronautical  Center 

• 

Regions 

0 

Scientific  and  Engineering  System 

Technical  Center 

o 

0 

9  Activity  fully  complies  with  federal  guidance  and  accepted  industry  practices. 

Q  Activity  partially  complies  with  federal  guidance  and  accepted  industry  practices — at  least 
one  element  is  performed, 

O  Activity  does  not  comply  with  federal  guidance  and  accepted  industry  practices. _ 
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Performance  Objectives 
Not  Maintained 


Performance  and 
Utilization  Not 
Consistently  Measured 


None  of  faa’s  seven  systems  fully  complied  with  federal  guidelines  and 
accepted  practices  for  establishing  current  performance  objectives,  faa 
established  performance  objectives,  such  as  response  time  requirements, 
for  some  of  its  automated  systems,  but  did  not  always  update  them.  In 
one  instance,  faa  never  established  performance  objectives. 

Although  an  important  component  of  establishing  performance  objec¬ 
tives  is  formulating  service-level  agreements,  faa  only  established  ser¬ 
vice-level  agreements  with  system  users  of  the  Aeronautical  Center 
system.  In  this  case,  faa  initiated  service-level  agreements  in  1985  on 
system  performance,  but  has  not  updated  them  since  then.  System  per¬ 
formance  is  now  managed  on  an  ad  hoc  basis  as  problems  arise;  user 
organizations  notify  faa  Data  Services  Division  staff  after  they  experi¬ 
ence  performance  problems. 


Federal  guidance  and  industry  practices  indicate  that  at  a  minimum, 
performance  and  utilization  data  should  be  routinely  collected  for  such 
components  as  central  processors,  memory,  and  input/output  subsys¬ 
tems.  However,  faa  does  not  consistently  collect  all  of  this  required  data 
for  any  system,  except  the  Common  System  at  the  Aeronautical  Center. 
For  this  system,  numerous  performance  management  tools  allow  the 
routine  collection  of  performance  and  resource  utilization  data.  Data  col¬ 
lected  include  the  number  of  jobs,  central  processor  utilization,  average 
execution  time,  and  average  printing  time. 

While  Common  System  performance  data  are  routinely  collected  at  the 
Aeronautical  Center,  they  are  infrequently  collected  at  regions.  Other 
than  a  one-time,  limited  faa  assessment  in  May  1991  to  evaluate  system 
performance  and  possible  system  upgrades,  each  region  monitors  its 
system  sporadically  and  does  not  consistently  collect  comprehensive 
data.  For  example,  at  some  regions,  central  processor  utilization  and 
disk  space  data  are  collected,  but  only  when  problems  are  expected  or 
have  occurred;  at  other  regions,  disk  utilization  data  are  routinely  col¬ 
lected,  but  processor  utilization  and  memory  data  are  compiled  only  on 
an  ad  hoc  basis. 

Data  collection  for  the  air  route  traffic  control  center  computer  systems 
is  inconsistent.  While  data  are  routinely  collected  on  processor  utiliza¬ 
tion  and  the  number  of  flight  plans  being  processed,  disk  and  memory 
utilization  data  are  not  collected  or  analyzed  regularly.  Without  this 
data,  FAA  does  not  know  if  disk  capacity  and  memory  utilization  are 
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nearing  maximum  limits,  thereby  posing  a  threat  to  continued  effective 
system  performance. 


Data  Analysis  and 
Reporting  Infrequent 


FAA  does  not  collect  comprehensive  data  on  its  systems’  performance 
and  utilization.  Further,  it  does  not  thoroughly  analyze  the  data  it  does 
collect.  For  example,  data  collected  on  system  performance  for  large 
TRACON  systems  are  sent  to  the  Technical  Center  and  headquarters,  but 
only  a  cursory  analysis  is  performed.  For  the  air  route  traffic  control 
center  computer  system,  a  wide  variety  of  data  are  collected,  but  only  a 
portion  are  analyzed.  For  example,  faa  analyzes  data  on  processor  utili¬ 
zation,  but  does  not  analyze  disk  capacity  or  response  time  data. 

By  contrast,  faa  has  analyzed  and  reported  performance  data  for  the 
Common  System  at  the  Aeronautical  Center  every  month  since  1979. 
Reports  containing  system  performance  data  from  the  past  4  months  are 
presented  monthly  to  Aeronautical  Center  officials.  The  reported  data 
are  subsequently  analyzed  at  monthly  meetings  where  problems  are 
identified  and  solutions  are  proposed,  such  as  modifying  hardware  and 
software  to  enhance  performance. 


Capacity  Planning 
Process  Is  Inadequate 


Except  for  the  recent  capacity  planning  activities  accompanying  the 
Computer  Resources  Nucleus  project,  capacity  planning  has  been  almost 
nonexistent  at  faa.  Several  one-time  studies  were  conducted  for  systems 
that  needed  to  be  upgraded  or  replaced,  but  faa  does  not  regularly  per¬ 
form  planning  activities.  The  following  table  summarizes  faa’s  compli¬ 
ance  with  the  capacity  planning  criteria  presented  in  chapter  2. 
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Figure  3.2;  Compliance  With  Capacity  Planning  Criteria 


Capacity  Planning  Activites 

Projecting  Future 

Work  Loads 

Proposing  Resources 

Preparing  Capacity 
Pians 

Air  Traffic  Control  Systems 

Air  Route  Traffic  Control  Center 

o 

o 

o 

New  York  TRACON 

Q 

d 

o 

Large  TRACONs 

d 

d 

o 

Small  TRACONs 

Q 

d 

o 

Common  System 

Aeronautical  Center 

• 

• 

d 

Regions 

O 

0 

o 

Scientific  and  Engineering  System 

Technical  Center 

o 

o 

( 

f) 

%  Activity  fully  complies  with  federal  guidance  and  accepted  industry  practices. 

O  Activity  partially  complies  with  federal  guidance  and  accepted  industry  practices— at  ieast 
one  element  is  performed. 

O  Activity  does  not  comply  with  federai  guidance  and  accepted  industry  practices. 


Future  Work  Loads  Not  Most  faa  systems  do  not  comply  with  federal  guidelines  and  accepted 

Adequately  Projected  industry  practices  for  projecting  future  work  loads.  Overall,  faa  does 

not  adequately  identify  when  new  or  modified  applications  will  be 
implemented,  collect  work-load  projection  data  from  users,  or  identify 
projected  modifications  to  performance  requirements. 

Future  work-load  projections  for  the  Common  System  at  the  Aeronau¬ 
tical  Center  were  last  prepared  in  1988,  based  upon  information  from 
the  Information  Resources  Management  Plan,  project  managers,  users, 
systems  analysts,  and  historical  data.  Although  this  information  pro¬ 
vided  useful  work-load  projections,  it  has  not  been  updated  since.  Fur¬ 
ther,  FAA  has  not  projected  Common  System  work  loads  at  the  regions 
except  for  the  analysis  to  determine  future  processing  requirements  for 
the  Computer  Resources  Nucleus  project. 

FAA  has  performed  only  minimal  work-load  projections  for  other  sys¬ 
tems.  For  its  air  traffic  control  systems,  faa’s  projections  of  future  work 
loads  have  been  based  on  limited  data.  For  example,  processor  utiliza¬ 
tion  and  system  work  loads  were  not  adequately  projected  in  deter¬ 
mining  system  alternatives  for  expanding  certain  large  terminal 
automation  systems,  faa  headquarters  personnel  indicated  they  asked 
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Technical  Center  personnel  for  future  capacity  requirements,  but  the 
Technical  Center  staff  did  not  know  how  to  project  these  requirements 
at  that  time.  As  a  result,  the  computer  memory  and  utilization  projec¬ 
tions  in  the  study  are  based  on  rough  estimates  from  site  personnel.  Rec¬ 
ognizing  that  it  needs  to  improve  in  this  area,  faa  is  planning  to  award  a 
contract  later  this  year  to  better  document  work-load  trends  and  predict 
future  performance. 


Future  Resources  Not 
Always  Proposed 


Only  the  Common  System  at  the  Aeronautical  Center  complied  with  all 
guidance  and  accepted  practices  for  proposing  future  resources.  For  its 
other  systems,  faa  did  not  identify  the  resources  required  to  meet 
expected  demand,  determine  a  schedule  for  updating  capacity  at  all 
sites,  or  study  alternatives  for  meeting  future  resource  demands. 

For  example,  large  terminal  automation  systems  are  undergoing  an 
interim  upgrade  to  sustain  operations  until  they  are  replaced  by  the 
Advanced  System  in  the  late  1990s.  However,  only  the  busiest  locations 
are  currently  being  considered,  and  future  system  work-load  require¬ 
ments  are  not  based  on  thorough  analysis.  As  a  result,  it  is  uncertain 
how  well  the  interim  upgrade  will  meet  faa  needs. 

Additionally,  future  resources  for  most  faa  systems  are  often  proposed 
after  a  capacity  problem  has  already  occurred.  For  example,  the  scien¬ 
tific  and  engineering  system  at  the  Technical  Center  lacked  sufficient 
capacity  to  meet  users’  needs  for  several  years  prior  to  an  faa  decision 
to  provide  additional  capacity  through  the  use  of  existing  or  planned 
computer  resources  at  the  Aeronautical  Center.  Future  resources  need  to 
be  proposed  early  enough  to  ensure  that  service  is  not  degraded  prior  to 
the  arrival  of  the  future  resources. 


Capacity  Plans  Not 
Prepared 


FAA  does  not  have  current,  complete  capacity  plans  for  any  of  its  sys¬ 
tems.  Some  planning  was  done  for  the  Common  System  in  preparation 
for  the  Computer  Resources  Nucleus  project.  However,  faa  has  not 
developed  strategic,  long-term  capacity  plans  for  all  sites,  reviewed  and 
updated  these  plans,  or  developed  tactical  or  short-term  capacity  plans 
for  any  of  its  systems  other  than  the  Common  System. 

A  strategic,  long-term  capacity  plan  was  prepared  for  the  Common 
System  at  the  Aeronautical  Center  in  1986  and  updated  in  1988.  How¬ 
ever,  the  plan  has  not  been  updated  since  that  time.  While  the  Computer 
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Resources  Nucleus  project  will  eventually  replace  this  system,  faa  offi¬ 
cials  indicated  that  the  Common  System  may  have  to  operate  at  least 
until  1995. 

In  1986,  FAA  began  developing  site-specific  capacity  plans  for  the  air 
route  traffic  control  center  computer  system.  These  capacity  plans  were 
designed  to  help  ensure  that  the  system  had  adequate  capacity  at  each 
site  for  future  air  traffic  work  loads,  operating  environment  changes, 
and  software  enhancements.  However,  faa  did  not  implement  site-spe¬ 
cific  capacity  plans.  According  to  an  faa  official,  the  air  route  traffic 
control  center  system  had  just  been  upgraded  at  that  time  and  managing 
capacity  was  not  a  priority  for  that  system. 

faa  also  did  not  develop  a  capacity  plan  for  the  scientific  and  engi¬ 
neering  system  at  the  Technical  Center.  As  a  result,  insufficient 
processing  capacity  has  degraded  system  performance  and  restricted 
the  number  of  users  the  system  can  accommodate. 
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faa’s  failure  to  implement  effective  performance  management  and 
capacity  planning  activities  agencywide  has  occurred  because  the 
agency  has  not  recognized  the  importance  of  capacity  management  as  a 
priority  activity.  As  a  result,  faa  organizations  responsible  for  capacity 
management  of  the  agency’s  automated  systems  have  not  provided  suf¬ 
ficient  guidance  on  how  to  implement  effective  capacity  management 
practices.  Further,  sufficient  expertise  is  not  available  to  perform 
capacity  management  activities.  Finally,  the  availability  and  application 
of  automated  capacity  management  tools  is  not  consistent  across  faa. 


FAA  Has  Not 
Implemented  Effective 
Capacity  Management 
Policies  and 
Procedures 


The  Associate  Administrator  for  Administration  and  his  Office  of  Man¬ 
agement  Systems  are  responsible  for  agencywide  information  resource 
oversight  and  policy-making  activities.  At  the  same  time,  faa  program 
organizations  retain  direct  management  responsibility  for  their  indi¬ 
vidual  systems.  For  example,  the  Aeronautical  Center  and  regions  are 
responsible  for  managing  the  Common  System,  while  Technical  Center 
officials  are  responsible  for  the  scientific  and  engineering  system.  The 
Air  Traffic,  Airway  Facilities,  National  Airspace  System  Development, 
and  System  Engineering  and  Development  divisions  are  responsible  for 
managing  air  traffic  control  automation  systems. 


None  of  these  organizations  have  implemented  effective  capacity  man¬ 
agement  policies  and  procedures,  faa’s  Office  of  Management  Systems 
has  not  developed  or  disseminated  any  capacity  management  guidance 
to  program  organizations  because  the  office  is  only  responsible  for 
system  oversight,  while  each  program  organization  is  responsible  for 
management  of  its  system,  according  to  that  Office’s  Information 
Resources  Management  Division  Manager.  Responsible  program  divi¬ 
sions  have  produced  some  capacity  management-related  studies,  notices, 
and  memos  over  the  past  several  years.  However,  in  most  cases  these 
were  generated  in  reaction  to  capacity  problems  and  did  not  comprehen¬ 
sively  describe  how  to  implement  performance  management  and 
capacity  plarming  activities.  For  example,  an  Aeronautical  Center  stan¬ 
dards  manual  discusses  performance  management  reporting  for  the 
Common  System,  but  no  capacity  planning  procedures  are  discussed. 
Without  comprehensive  policies  and  procedures,  capacity  management 
practices  are  either  inconsistent  or  not  performed  at  all. 


In  one  instance,  faa  did  develop  detailed  capacity  management  policies 
and  procedures,  but  they  were  never  implemented.  A  capacity  manage¬ 
ment  plan  was  published  in  June  1986  for  faa’s  air  route  traffic  control 
center  computer  system.  A  more  detailed  document  defining  specific 
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computer  capacity  management  procedures  was  later  completed  in  Feb¬ 
ruary  1988.  These  documents  described  sound  capacity  management 
policies  and  procedures  that  would  ensure  that  capacity  and  response 
time  requirements  were  satisfied  over  the  life  of  the  system.  However, 
according  to  faa  officials,  because  no  capacity  problems  at  air  route 
traffic  control  centers  occurred  at  that  time,  the  program  was  not  a  pri¬ 
ority  and  was  never  implemented.  As  a  result,  only  a  small  portion  of 
the  performance  data  identified  in  the  plan  is  currently  measured  and 
analyzed.  For  example,  disk  capacity  and  response  time  data  are  not  col¬ 
lected  and  analyzed. 


To  plan,  administer,  and  evaluate  capacity  management  programs,  agen¬ 
cies  need  to  designate  personnel  who  are  skilled  in  performance  manage¬ 
ment  and  capacity  planning  techniques;  familiar  with  tools  that  can 
collect,  analyze,  and  report  data;  and  knowledgeable  about  the  daily 
operations  of  the  systems.  Private  sector  organizations  have  such  per¬ 
sonnel  because  they  have  made  the  capacity  management  function  a  pri¬ 
ority  and  devoted  the  necessary  resources  to  it. 

None  of  faa’s  organizations  responsible  for  capacity  management  have 
assigned  in-house  or  contract  staff  to  develop  a  capacity  management 
program.  Further,  except  for  one  staff  person  at  the  Aeronautical 
Center,  none  of  faa’s  organizations  have  dedicated  staff  to  carry  out 
computer  capacity  management  activities.  For  example,  at  faa’s  central 
region,  the  staff  person  assigned  to  capacity  management  duties  indi¬ 
cated  she  only  spends  about  6  percent  of  her  time  on  capacity  manage¬ 
ment  activities  since  there  have  not  been  capacity-related  problems. 

Without  sufficient  expertise,  the  effectiveness  of  capacity  management 
activities  is  limited.  For  example,  the  analyst  assigned  to  capacity  man¬ 
agement  at  the  Aeronautical  Center  stated  that  performance  tools 
cannot  be  fully  used  because  staff  are  not  available  to  operate  them. 

This  is  supported  by  a  consultant  hired  by  faa,  who  concluded  that  the 
number  of  capacity  management  staff  in  place  was  inadequate  and 
lacked  necessary  expertise.  In  comparison,  several  private  organizations 
have  a  significant  work  force  dedicated  to  capacity  management  activi¬ 
ties  (see  App.  I). 


FAA  Lacks  Capacity 
Management  Expertise 
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To  assist  in  capacity  management  activities,  performance  management 
and  capacity  planning  tools  are  often  used  to  collect  data.  These  tools 
are  used  to  produce  reports  that  help  analyze  daily  activity,  project 
computer  capacity,  identify  system  inefficiencies  and  malfunctions,  and 
improve  response  time.  For  example,  organizations  may  employ  auto¬ 
mated  tools  to  indicate  automated  resource  consumption  by  a  depart¬ 
ment  over  a  period  of  time. 

The  use  of  these  automated  tools  varies  greatly  among  faa’s  automated 
systems.  Some  systems  are  supported  by  numerous  software  products 
that  collect,  analyze,  predict,  and  model  computer  performance  and 
planning  data,  while  other  systems  are  not  supported  by  any  automated 
tools.  For  example,  staff  maintaining  the  Common  System  at  the  Aero¬ 
nautical  Center  have  several  operating  system  facilities  as  well  as  com¬ 
mercial  software  products  to  help  them  measure  computer  resource 
utilization  and  to  determine  when  capacity  will  be  exhausted.  Con¬ 
versely,  performance  of  faa’s  smaller  terminal  automation  systems  has 
not  been  monitored,  although  faa  is  developing  a  performance  monitor 
and  plans  to  provide  it  to  smaller  tracons  in  1992. 

faa’s  use  of  automated  tools  is  also  inconsistent  in  that  different  sites 
operating  identical  systems  do  not  always  use  the  same  performance 
monitoring  tools.  For  example,  staff  supporting  the  administrative 
system  at  the  Technical  Center  use  two  commercial  tools  to  help  manage 
disk  storage  space,  while  staff  maintaining  the  same  system  at  the  Aero¬ 
nautical  Center  are  not  using  any  such  tools.  Because  capacity  manage¬ 
ment  guidance  is  not  available,  there  is  no  method  to  ensure  the 
consistent  use  of  computer  capacity  management  tools  for  faa’s  admin¬ 
istrative  computer  systems. 
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Conclusions  ^  comprehensive  computer  capacity  management  program  is  essential 

to  the  successful  operation  of  faa’s  major  automated  systems.  Both  per¬ 
formance  management  and  capacity  planning  activities  must  occur  regu¬ 
larly  to  ensure  that  computer  systems  have  sufficient  capacity  to  meet 
both  current  and  future  mission  needs.  Private  organizations  have  dis¬ 
covered  that  sound  capacity  management  activities  are  essential  to  their 
success  in  a  competitive  business  environment. 

Although  FAA  has  improved  some  of  its  capacity  management  practices, 
it  still  has  not  implemented  an  effective  computer  capacity  management 
program  for  its  major  automated  systems.  Without  such  a  program, 
faa’s  capacity  management  practices  are  inconsistent  and  incomplete. 
Due  to  inadequate  and  inconsistent  performance  monitoring,  faa  does 
not  know  if  its  systems  are  working  efficiently  and  effectively.  Simi¬ 
larly,  capacity  planning  for  most  systems  seldom  occurs,  resulting  in 
uncertainty  about  when  existing  systems  will  need  to  be  replaced  or 
upgraded. 

Because  capacity  management  is  not  a  priority  at  faa,  there  is  a  lack  of 
adequate  policies,  procedures,  expertise,  and  tools.  In  one  instance 
where  a  sound  comprehensive  capacity  management  plan  was  devel¬ 
oped  for  air  route  traffic  control  center  computers,  it  was  never 
implemented. 

Because  the  Common  System  and  current  air  traffic  control  systems  will 
function  for  many  more  years,  faa  needs  to  address  these  deficiencies.  It 
is  especially  critical  that  faa  implement  an  effective  capacity  manage¬ 
ment  program  for  its  air  traffic  control  systems,  given  their  crucial  role 
in  ensuring  aviation  safety.  Otherwise,  faa  will  continue  to  react  to 
serious  capacity-related  problems  after  they  occur,  instead  of 
preventing  them  before  they  do. 


RGCOmmCndationS  recommend  that  the  Administrator,  faa,  develop  computer  capacity 

management  policies  and  procedures  for  air  traffic  control  systems  and 
the  Conunon  System.  These  policies  and  procedures  should  require 
implementation  of  capacity  management  programs  for  each  system, 
which  include  establishing  performance  objectives,  measuring  perform¬ 
ance  and  resource  utilization,  analyzing  and  reporting  capacity  manage¬ 
ment  data,  projecting  future  work  loads,  proposing  resources  to  meet 
future  demands,  and  preparing  capacity  plans.  In  doing  this,  the  Admin¬ 
istrator  should  place  highest  priority  on  implementing  capacity  manage¬ 
ment  programs  for  systems  that  are  most  critical  to  aviation  safety.  In 
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addition,  we  recommend  that  the  Administrator  develop  or  obtain  the 
capacity  management  expertise  necessary  to  produce  policies  and  proce¬ 
dures  and  administer  capacity  management  programs.  These  personnel 
should  be  skilled  in  performance  management  and  capacity  planning 
techniques,  familiar  with  tools  available  to  collect,  analyze,  and  report 
data,  and  knowledgeable  about  the  daily  operations  of  the  systems. 

Finally,  we  recommend  that  the  Administrator  develop  or  acquire  auto¬ 
mated  performance  management  and  capacity  planning  tools  for  each 
system  and  consistently  use  them  to  assist  in  the  uniform  collection, 
analysis,  and  reporting  of  performance,  utilization,  and  capacity  data. 
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Farmland  Industries 

This  agricultural  cooperative,  headquartered  in  Kansas  City,  Missouri, 
employs  approximately  7,000  and  generates  over  $3  billion  in  yearly 
sales.  Farmland  has  six  primary  areas  of  business;  petroleum  products 
and  services,  fertilizers,  foods,  feeds,  financial  services,  and 
transportation. 

Farmland  Industries  maintains  a  variety  of  automated  systems  ranging 
from  a  large,  corporate  mainframe  to  a  network  of  microcomputers  in 
field  locations.  In  addition,  the  company  maintains  approximately  6,000 
remote  terminals  in  19  states  to  provide  agricultural  communities  with 
access  to  company  data  bases  and  information  services.  The  company 
employs  diverse  software  applications,  such  as  commodity  news  ser¬ 
vices;  credit  cards  for  gas  pump  operations;  and  payroll,  accounting, 
warehousing,  and  transportation  support. 

Capacity  Management  at 
Farmland  Industries 

Farmland’s  current  capacity  management  team  consists  of  a  manager,  a 
performance  specialist,  and  a  capacity  planner.  Farmland’s  managers 
point  out  that  their  capacity  management  team  is  a  force  in  the  com¬ 
pany’s  management  and  not  simply  a  reporting  mechanism.  The  team 
prepares  a  strategic  capacity  plan,  which  serves  as  an  18-month 
blueprint  for  Farmland’s  automated  activities. 

Performance  Management 

In  carrying  out  its  performance  management  activities.  Farmland  tracks 
system  availability  by  each  subsystem  and  monitors  performance 
indicators,  such  as  processor  utilization,  disk  paging  rates,  and  response 
times  for  users.  These  various  system  indicators  are  used  to  tune  the 
system  and  to  construct  performance  reports,  which  feed  into  the  com¬ 
pany’s  overall  strategic  capacity  plan.  Also,  Farmland  has  established 
service-level  objectives  in  consultation  with  system  users  and  is  cur¬ 
rently  developing  service-level  agreements  with  its  customers  who 
purchase  information  services. 

Farmland  uses  many  automated  software  tools  to  help  collect  perform¬ 
ance  data,  which  gives  managers  more  time  to  analyze  performance 
data.  In  addition,  the  company  has  developed  data  bases  to  help  manage 
the  collected  performance  data. 

Capacity  Planning 

To  prepare  capacity  plans.  Farmland  capacity  managers  use  both  per¬ 
formance  data  and  work-load  projections.  To  properly  project  future 
work  loads  and  new  applications,  managers  measure  existing  work  loads 
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and  interview  users  to  determine  future  resource  requirements. 
According  to  a  Farmland  official,  such  planning  gives  Farmland  the  flex¬ 
ibility  it  needs  to  study  various  ways  to  meet  future  capacity  demands. 


Co  via 


Covia  traces  its  roots  back  to  the  Apollo  Services  Division  of  United  Air¬ 
lines.  In  1987,  Covia  became  an  independent  subsidiary  of  United  Air¬ 
lines.  While  United  Airlines  still  owns  60  percent  of  this  company,  Covia 
functions  as  an  information  service  bureau  for  many  travel-related  busi¬ 
nesses,  including  seven  major  airlines. 

Although  Covia  markets  a  wide  range  of  travel-related  products,  its 
major  business  is  airline  reservations.  Covia’s  3,000  employees  provide 
information  and  reservations  to  over  15,000  travel  agency  subscribers 
in  42  countries  through  its  Apollo  System.  In  addition,  the  Apollo 
System  makes  reservations  for  120  hotel  chains,  19,000  hotel  properties, 
and  37  rental  car  companies. 

Covia’s  automated  data  processing  and  telecommunications  network  is 
composed  of  a  vast  array  of  hardware,  software,  and  telecommunica¬ 
tions  equipment.  Currently,  Covia’s  automated  data  processing  and  tele¬ 
communications  assets  include  22  mainframe  computers,  65,000 
terminals,  30,000  ticket  printers,  and  2,600  data  circuits.  The  system 
processes  1,400  entries  per  second  and  approximately  6  million  airfare 
changes  per  day. 


Capacity  Management  at 
Covia 


Covia’s  information  resources  require  careful  management  to  stay  com¬ 
petitive  within  the  information  services  industry.  Thus,  the  capacity 
management  organization  plays  an  essential  role  in  the  company’s  suc¬ 
cess.  A  system  performance  and  capacity  planning  group  is  located  in 
each  of  Covia’s  three  major  data-processing  areas:  host;  network;  and 
microcomputer.  The  host  group  contains  12  full-time  employees  dedi¬ 
cated  to  system  monitoring  and  capacity  planning,  while  the  network 
group  has  20  full-time  employees  dedicated  to  performance  manage¬ 
ment,  capacity  planning,  and  network  design.  The  microcomputer  group 
consists  of  six  full-time  employees  dedicated  to  microcomputer  capacity 
planning  and  local  area  network  performance  monitoring. 


Performance  Management 


Covia  monitors  system  performance  by  tracking  daily  and  monthly  per¬ 
formance  against  stated  objectives.  The  reliability  of  Covia’s  host 
system,  for  example,  is  measured  in  terms  of  daily  system  errors  and 
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system  availability  is  measured  monthly.  Response  time  is  an  important 
performance  indicator,  and  Covia  tracks  both  host  response  time  and 
network  response  time,  since  both  affect  total  users’  response  times. 

Other  system  parameters  measured  to  track  performance  against  stated 
goals  include  central  processing  utilization,  input/output  rates,  and 
amount  of  available  storage.  Measurements  are  taken  every  minute 
throughout  the  24  hours  of  system  operation.  In  addition,  special  moni¬ 
toring  is  performed  for  the  input/output  subsystem.  During  peak  usage, 
the  input/output  system  is  analyzed  to  track  heavily  used  records  and 
devices.  This  extensive  performance  management  allows  the  company 
to  keep  each  system  running  as  efficiently  as  possible.  According  to  a 
Covia  official,  this  results  in  higher  levels  of  customer  service  and 
decreased  costs  for  Covia  and  its  many  customers. 


Capacity  Planning  Covia’s  capacity  planning  entails  many  activities.  Covia’s  work-load 

forecasting  is  based  on  an  analysis  of  historical  trends,  as  well  as  knowl¬ 
edge  of  related  economic  forces  such  as  changes  in  airline  fares.  Work 
loads  are  analyzed  monthly  to  identify  changes,  and  the  company’s  per¬ 
formance  group  is  required  to  evaluate  the  impact  of  new  projects  on 
system  performance  and  capacity. 

Covia  also  constructs  18-month  capacity  plans  for  each  of  its  systems. 
These  plans,  updated  quarterly,  include  data  on  projected  available 
capacity,  additional  hardware  needs,  and  planned  major  software  appli¬ 
cations.  For  Covia’s  network  systems,  capacity  needs  are  analyzed  and 
actions  are  scheduled  for  each  site  to  accommodate  sites’  unique 
requirements. 


Yellow  Freight 
Systems  Inc.  of 
Delaware 


Yellow  Freight  Systems  Inc.  of  Delaware  is  a  holding  company  that  pro¬ 
vides  freight  transportation  services  through  its  subsidiaries.  One  of  the 
largest  freight  carriers  in  North  America,  Yellow  Freight  serves  more 
than  300,000  domestic  and  overseas  customers.  Its  core  business  is  the 
consolidation,  transportation,  and  redistribution  of  freight,  utilizing  a 
single  network  of  drivers,  terminals,  and  equipment.  The  company 
employs  approximately  30,000  people  located  at  more  than  630  business 
locations. 
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Yellow  Freight  maintains  a  variety  of  automated  systems.  The  company 
recently  developed  additional  computerized  services  to  expand  its  busi¬ 
ness  base,  as  well  as  reduce  costs  and  increase  efficiency.  The  com¬ 
pany’s  new  EDI  Partners  system,  for  example,  can  assist  customers  with 
shipment  tracing,  billing,  cash  management,  and  planning  functions. 
Another  new  system — SVSNET — allows  the  company  to  obtain  informa¬ 
tion  about  efficient  loading  patterns  and  the  most  cost-effective  routes. 
Yellow  Freight  is  also  replacing  its  manual  data  entry  operations  with  a 
bar  coding  system  to  improve  quality  control  and  increase  the  amount  of 
usable  management  information,  and  has  a  pilot  project  involving 
mobile,  in-cab  computers  designed  to  transmit  shipment  information 
faster. 


Capacity  Management  at 
Yellow  Freight 


Yellow  Freight  employs  a  decentralized  capacity  management  approach. 
Each  system  group  within  the  company’s  Management  Information  Ser¬ 
vices  Division  has  a  capacity  management  team  consisting  of  a  full-time 
manager  and  one  or  two  other  employees  with  part-time  capacity  man¬ 
agement  duties.  Capacity  management  reports  completed  by  these  teams 
are  distributed  to  directors  and  vice  presidents  in  the  company’s  Infor¬ 
mation  Services  Division.  In  addition,  key  personnel  regularly  review 
performance  management  results  and  upcoming  capacity  planning 
issues. 


Performance  Management  Yellow  Freight’s  capacity  management  teams  collect  a  variety  of  com¬ 
puter  performance  data,  including  internal  response  time  by  transaction, 
network  response  time,  central  processor  utilization,  processing  time  by 
region,  and  batch  time  by  system.  Software  packages  are  used  to  collect 
data,  build  data  bases,  and  analyze  systems’  data. 


Capacity  Planning 


Performance  data  and  work-load  projections  are  used  to  construct 
future  hardware  and  software  requirements,  which  are  used  to  produce 
a  capacity  plan.  Capacity  planning  at  Yellow  Freight  considers  the 
impact  of  (1)  expected  growth  in  transaction  volume,  (2)  additional 
software  applications,  and  (3)  enhancements  to  existing  software  appli¬ 
cations.  According  to  a  company  representative,  this  planning  process 
has  significantly  improved  response  times  for  users  and  prevented  the 
occurrence  of  sudden  capacity  problems. 
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MCI  Corporation 

MCI  Corporation  is  one  of  the  leading  companies  in  the  global  telecom¬ 
munications  services  market,  where  change  is  often  dramatic  and  fast- 
paced.  MCI’s  24,500  employees  strive  to  work  within  this  dynamic  envi¬ 
ronment  to  provide  telecommunications  services  to  residential,  commer¬ 
cial,  and  government  customers. 

Capacity  Management  at 
MCI 

MCI  has  a  capacity  management  team  for  each  of  its  two  major  informa¬ 
tion  systems  divisions.  For  example,  the  Data  Center  Operations  Divi¬ 
sion  capacity  management  team  consists  of  seven  full-time  employees 
and  a  team  manager,  with  the  team  manager  reporting  directly  to  the 
Vice  President  for  Management  Information  Systems  Operations.  The 
team’s  primary  task  is  long-term  capacity  planning. 

Performance  Management 

MCI  establishes  performance  objectives  for  such  indicators  as  response 
time,  reliability,  and  system  availability.  Some  of  these  indicators  are 
recorded  and  reported  daily  while  others  are  reported  on  an  exception 
basis.  For  example,  performance  objectives  related  to  MCI’s  telephone 
network  are  reported  daily  since  these  are  crucial  to  the  business.  Like 
many  other  successful  capacity  management  programs,  MCI  uses  auto¬ 
mated  software  tools  to  collect  performance  data.  Currently,  software 
tools  collect  performance  data  on  the  central  processor,  memory,  and 
the  input/output  subsystem. 

Capacity  Planning 

MCI  measures  existing  work  loads  and  projects  future  work  loads  as 
part  of  the  company’s  capacity  planning  process.  Changes  in  work  load 
are  forecasted  on  a  quarterly  basis  by  reviewing  historical  trends  and 
considering  new  user-identified  software  applications.  After  future 
work  loads  have  been  forecasted,  resource  requirements  are  constructed 
and  alternative  approaches  to  meeting  these  requirements  are  identified. 
MCI  prepares  a  5-year  strategic  capacity  plan,  updated  annually,  which 
includes  a  tactical  section  focusing  on  the  next  2  years. 
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